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ABSTRACT 


Several recent studies of Naval Communications have 
determined that the system is not operating properly. This 
paper presents a test procedure which will improve 
the performance of the equipment that composes the 
Communications System. The need for a test procedure, modes 
of failure and present test procedures are examined. It is 
determined that present test procedures referred to as system 
level tests are mislabelled and are actually subsystems level 
tests. A Systems Level Test, the Standard Measurement Tech- 
nique (SMT) which applies inputs of known degradation to a 
system and quantifies the output is presented. Present 


problems and potential applications of SMT are discussed. 
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I. COMMUNICATIONS AVAILABILITY AND THE NEED FOR A SYSTEM 

TEST PROCEDURE 

In recent years several studies of the Naval Communica- 
tions System have been conducted such as the Base Line II, 
(1), 1966, Rope Eval exercise, 1971, (2) Mollohan report, 
1972, (3) and the CNO's Industrial Advisory Committee on 
пее отидат саетове (СТАСТ) 911 (4). 1 повећа intent 
of this paper to restate the studies which have been made but 
rather to show how these studies support the need of a com- 
munications system test procedure which will perform better 
than our present subsystem level tests. 

Many of the deficiencies uncovered during one study 
(which is classified and cannot be identified in order to keep 
this paper unclassified), were attributable to the fact that 
an adequate method of determining and monitoring a Communica- 
tion System's Performance was not available. The inability 
to locate the cause of the failure or degradation would very 
often result in placing the blame on the other station (s) or 
on the atmosphere. An туза. of Black Magic was introduced 
because neither end of the trunk knew where to assign the 
blame, (itself, the other station or the atmosphere). There 
was no method available to determine the system's level of 
performance. There are standards available by which to 
measure component, equipment and subsystem performance but 
there are no standards to measure the overall performance of 


a system either shipboard or shore station. The Report said 





there is a need to prepare and publish detailed standard 
procedures for operations, maintenance and management of 
Naval Communications on a systems level. The Report also 
pointed out that the "systems approach" is not used or taught 
at any level of the Naval training establishment. 

The studies mentioned have attempted to measure the Naval 
Communications System's ability to communicate and the con- 
clusions are that the system is a very poor performer. It 
takes too much time, uses too many assets, looses messages 
and, occasionally, ceases to function. 

The CIACT report called upon the Navy to develop an over- 
all plan to include a general performance description for the 
total Naval Communications system [4, р. 17]. It can easily 
be argued, that some means must be available to measure the 
"general system performance" or the "description" is of 
little value. This implies the need for a system test pro- 
cedure. The CIACT goes on to say the Naval Communications 
Command 1S moteallowed billets for communications/electronics 
engineers to Boerne service and manage a complex communica- 
tions system. With no person ready or trained to manage a 
communications system, system accountdbility was not called 
for, therefore, there was no test of the system in order to 
measure it. 


Now that official attention has been drawn to the need of 
a System test procedure to measure the level of performance 
of the system, an attempt will be made to specify what the 


system test procedure should accomplish. 





Тһе test procedure should be able to determine if the 
total system is operating and if so, at what level. No 
system will operate continuously at 100% and if System A is 
operating in a degraded mode, will it be able to communicate 
with System B which is also operating in a degraded mode? 

How much degradation can the entire system accept and still 
perform satisfactorily? The test procedure must be able to 
determine how much degradation (either in the input signal, 
or within itself), a system can tolerate. The test procedure 
should be able to predict the system availability so that the 
user can plan the execution of this assigned mission. 

NASA has established long-term average system performance 
goals for the various transmission media (microwave, land 
line, submarine cable, communications satellite, and HF radio). 

NASA circuit availability objectives expressed on SITE 
ages, indicating the portion of the total scheduled operating 
time during which the circuit is expected to be available. 

Microwave and landlines 99.5$ 

Submarine cables 99252 

Communications Satellite 99,8% 

High-frequency radio 95.0% 

Table 1-1 NASA circuit availability; adapted from 
Stelter [5, p. 274] 

шпесе соа а рктпсірЛефуипавс Ене, СТАСТ герогк са115 Тог, 
are expressed as percentages, indicating the portion of the 
Toralsscheduled operating time during which the circuit is ex- 


pected to be available to the user. "Circuit: asi used here. 





includes the medium of transmission and the equipment at 

both ends; a trunk. NASA measures its circuit performance 

by subjecting it to a bit error rate test of 24 hours duration 
on a periodic schedule. А Data Transmission Test Set gener- 
ates a 2047 bit pseudo random pattern that is transmitted via 
the modems normally associated with a particular circuit, to 
a compatible receiver.: The receiver is attached to a Print- 
Punch recorder. This recorder provides a time-tagged, perma- 
nent record of the circuit measured bit error rate. In this 
manner a circuit level of performance is established [5, p. 
2121. 

Compiling circuit performance levels enables NASA to 
establish a system level of performance. 

A Naval Underwater System Center study has proposed a 
Ба: фак test procedure. The test is made by applying an in- 
put sequence X such that the resulting output sequence will 
бей вала ов, the Circult 1s opelvatbing correctly. 

In this manner a detailed knowledge of the logic elements, 
the mechanics, the theory and their interaction used to im- 
plement the system is ae necessary in order to test it. The 
test will detect almost all malfunctions resulting from 
single, simultaneous or unanticipated failures. [6 , p- Mij.: 

The final report on Evaluation of the Unencrypted Data 
Link System of the P-3C airplane (INSERV REPORT) brings out 
the need for a system test procedure which can isolate fault 
со = сопролеле It Criticizes the AN/ACO-5 built-in test 
equipment which isolates faults to a group of modules or sub- 


systems which can contain up to 46 components. р. 231 
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The test procedure Te be able to locate the cause of 
the degradation or failure down to the component or piece of 
equipment level in order to ensure maintainability and high 
system availability. 

In summation, the Communications System test procedure 
should be able to accomplish the following: 

1) Detect any significant malfunction. 

2) Quantify the level of performance or determine the 

level of degration at which the system is operating. 

3) Determine how much degradation in the input Signal a 

system can tolerate and still perform satisfactorily. 

4) Locate the cause of the malfunction down to the 

component piece of equipment level. 

5) Aid in improving system maintainability. 

A test procedure which attempts to satisfy these criteria 
is presented in this paper. Following chapters will attempt 


to show why a test procedure is needed, what a test should 


measure and how various present test procedures compare. 
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tia Ae STUDY «OF AVAILABILITY 


The ultimate goal of any system or equipment is to fulfill 
the particular mission for which it was designed. Before any 
system or equipment can fulfill its mission, it must be avail- 
able. Increased equipment complexity, new performance re- 
quirements, and extreme environments have resulted in higher 
failure rates, greater requirements for maintenance and lower 
availability of current systems. [8, р. 117]. Тћеке has been 
a steady improvement in component reliability, but ағ а slower 
rate than the increase in complexity. So systems and equip- 
ment reliability has fallen to the point where the system 
often is not available to the user when and where it is re- 
quired. Furthermore, the systems and equipment are so ex- 
pensive that it is no longer practical to get around this 
problem by buying more systems than are needed. The operators 
have come to realize that the systems they have, must be avail- 
Е вже п ока gr ul Fi rhe l rm particular mission. Па ри 
221], 

In the 50's and early 60's system availability was synon- 
ymous with reliability. The prevailing attitude was that if 
enough reliability could be built into a system its correspond- 
ing availability would ensure system effectiveness. This 
attitude was fostered by NASA because it determined that in 
Space work there would be no means to replace or maintain any 
со реп краси nad failedame NASA utilized redundancy of critical 


components to attain the needed level of reliability. 
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Therefore reliability was equal to availability and all of 

the literature of the period stresses this point. As systems 
engineers have learned to understand and work with reliability 
they have realized that the initial premise is true but that 
systems availability could be obtained by means other than 
reliability such as maintainability. 

Tt must be recognized that systems availability not reli- 
ability is the ultimate criteria and that it can be obtained 
through reliability and maintainability. This criteria can 
be expressed mathematically as the relationship between 
availability A, ENDE: time between failure (MTBF) 
and maintainability; mean time to repair (MTTR). UT Ner SIE 

А = MTBF 
MTBE METR 
This paper supports that premise and will attempt to show that 
adequate testing is needed to support maintainability in order 


to achieve availability. (Fig. 2-1) 


A. RELIABILITY 

Today, although an environment of declining "technology 
spending", especially related to defense and aerospace, empha- 
Sis is focused on operational and system effectiveness. Aus- 
terity in spending, coupled with the need for more sophisticated 
equipment, focuses an even more important light on the need 
for a new and more practical reliability engineering technology. 
The continued criticality of effective equipment performance in 


sophisticated systems creates a changing emphasis on reliability 
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and requires much more careful control of the design process. 
ШИРЕКТЕ 

Reliability definition: 1) The ability of an item to 
perform a required function under stated conditions for a 
Stated period of time. 2) The characteristics of an item 
expressed by the probability that it will perform a required 
function under stated conditions for a stated period of time 
[12, p. 198]. This general definition is just a starting 
Boint; the product specification for each product should in- 
clude clear definitions of all modes of operation important 
to the function of the device. Dormant and operating environ- 
ments as well as life expectancy are necessary considerations 
in the specification. 

A satisfactory definition of reliability in the product 
specification should provide all the information necessary to 
judge whether device operation is proper or improper. The 
definition should state what constitutes failure and the con- 
ditions under which failure occurs. Reliability is usually 
stated as the "mean C between failure" - MTBF along with a 
confidence interval on the mean time. 

In order P. ensure that a system has the required reliabil- 
ity, DOD has found it necessary to establish a formal, four 
phase, system life cycle. The system consists of Concept 
bBormwlrronoscontract Definition, Acquisition and Operational 
phases. 

in tHe?Concept Formulation phase, the mission of the system 


is justified and its long range goals are stated. The mission 
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is defined as reliability requirements predictions are made 
on the basis of the functions that must be performed to 
accomplish system effectiveness and mission successes. The 
accuracy of these predicted requirements need only be enough 
to justify proceeding with the next phase of the project. 
Alternative solutions are listed and feasibility studies may 
be conducted. 

The Contract Definition phase includes identification 
and specification of system characteristics and the prepa- 
ration of program plans which will assure a high probability 
of achieving the specified missions in a cost effective 


manner. Modern program management demands that specification 


AVAILABILITY 
RELIABILITY MAINTAINABILITY 
(Operating time) (down time) 
TEST PROCEDURE 
AVAILABILITY 


- Adapted from Blanchard (8, р. 18) 


(Fig. |) 
decisions be based on quantitative analysis. Numerical 
values for the system characteristics, 1.e., reliability, 


шатпоси пара ам, operability, safety, cost analysis, quality, 


security, training, procedures and logistics, must be predicted 
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during the Definition phase of the system life Зе. Reli- 
ability and maintainability program plans must be part of a 
total management system. Identification and funding of reli- 
ability and maintainability programs must be an integral part 
of a program managers overall cost, schedule and technical 
control system.  [13, p. 1-5]. These characteristics must be 
repredicted or assessed repeatedly throughout the Acquisition 
phase and they must be measured or demonstrated before and 
during the Operational phase. These numerical values have 
value only when used to give a predicted, assessed or пеа- 
sured value of overall system effectiveness. 

While determining a quantitative reliability specification, 
certain factors must be kept in mind. It is impossible for 
man to build a system with 100% reliability because of the 
limits of knowledge, physical laws and monetary constraints. 
There will be a reliability decrement due to design defi- 
Cliencies, manufacturing deficiencies and use. (Fig. 2-2) 
These losses will occur. In addition to the physical environ- 
ment problem the effect of the social environment containing 
people, the m mut intrisee, and their expectations, 
goals, motivations, training, and other psychological and 
physiological attributes will introduce some variance to the 
stated reliability.  [15, p. 255]. The system planners must 
determine how much reliability loss can be expected and how 
much can be tolerated. There are numerous mathematical 
methods of quantifying the reliability necessary for a par- 
ticular system. Such as Monte Carlo models [16, p. 4) and 


regression analysis. 
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An additional input needed before a level of reliability 
sought can be selected is cost. It must be recognized that 
there is a cost effective level of reliability and that any 
demand for greater reliability will be disproportionate with 
its cost. Above this point some means to achieve increased 
availability other than reliability should be sought.  Trade- 
offs between cost, availability, reliability and maintain- 
ability must be analyzed to determine the cost effective 
blend. 

With quantitatively designed reliability values available 
manufacturing can be monitored to ensure attainment of the 
needed reliability levels. Emphasis must be placed on the 
fact that the reliability goals are determined prior to 
manufacturing and are included in the specifications. In 
this manner all personnel working on, and the ultimate users 
Св the тіреп аке aware OF the established relifability levels 
and can work to assure their attainment. This assumes the 
existence of 1) testing or verification procedures to deter- 
mine the actual level of reliability and 2) management control 
procedures to handle en variance between the desired and 
actual levels of reliability attained. The management con- 
trol procedures should be specified in a management plan. 
While reliability considerations can be very useful they are 
not a panacea as the following examples will demonstrate: 

1) When using MTBF as a reliability criteria how is a 

component which does not fail but goes into degraded 


operation handled. It is not a black and white but 
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2) 


3) 


a grey area. If specifications were written to cover 
all grey areas they would be voluminous. 
Determining reliability and cost estimates for 
marginal analysis 1S expensive and time consuming. 
At present the performance of some systems is difficult 
to quantify and does not lend itself to quantitative 
values which can be utilized for reliability specifi- 
cations; 1.e., voice communications systems. The | 
object of the system is to pass intelligible infor- 
mation. How is "intelligible" defined?  MIL-STD-188C 
revised 24 Nov. 1969 gives the following performance 
objectives. "Tactical transmission system volume, 
loop losses, switch losses and inter-office trunks 
shall be engineered to provide the following listen- 
ers preference ratings in terms of listeners prefer- 
ences: 

85% of the users will rate the circuit "Good". 

10% of the users will rate the circuit "Fair". 

5% los спе users will rate tHe circuit “Poor or 


Поссе ИЕ р. «0f 


Although these are performance criteria, they do not satisfy 


the criteria of quantitative, reliability specifications. 


MAINTAINABILITY 


Ка алдар Ша дети ЕО А t-imeMftuncuon, the prob- 


ability, understated conditions, that a maintenance process 
will terminate successfully within a stated time measured 


rom xnritratron of the process. 
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The maintenance need can be brought on by degraded 
Operation, catastrophic failure or because of the lapse 
of a predicted reliability time period. The maintenance 
Шессесіз can be preventative maintenance which considers time 
as a constant, either because regular periods of fixed 
duration are provided for the purpose or because the tasks 
are so routine that performance times exhibit negligible 
dispersion. On the other hand corrective maintenance typi- 
cally involves unforseen contingencies such as catastrophic 
failure or systems deterioration while still performing above 
a "failed" level (18, р. 137). | 

Reliability is one side of the system "availability" 
coin. Though complex equipment can fail, reliability has 
attained a degree of sophistication that makes it possible to 
predict failure rates with a fair degree of accuracy. The 
other side of the coin, maintainability, presents a problem. 
Even though the need for maintainability has been recognized 
in some quarters it has not received wide attention and 
therefore has not reached the stage where it can be effec- 
tively quantified with м appreciable degree of acceptance. 
Since most things are fallible, maintainability, or the need 
Чотой fact of life. @wWhen failure occurs repair is 
required and the faster the better. If either "ability" 
а поток сок се јобБ ОЕ гаг ПЫ čto support the other, avail- 
abi брала Се го 19, р. 1-5]. 

шде ссесетаргаоп ой ауалтартілісу це contained in 


NAVSHIPS 94324. The mathematical approached used is 
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A =R + M (1-R) 
where A is the availability, R is the equipment reliability 
and Mo is the operational maintainability of the equipment. 
ПРИ а В |. 

Тһе initial Concept Formulation phase, when a future 
system is being considered and the alternatives are being 
examined, 15 when maintainability long range goals should 
first be considered. From the point in time when reliability 
becomes a consideration maintainability must also be present. 
In the Concept Formulation phase, maintainability functions 
such as, logistics, support systems, personnel and training 
must be present and kept visible throughout the system 
evolution. 

Maintainability requires an integration of design and 
maintenance engineering effort to provide controls which 
assure that the system being designed is adequately, expedi- 
tiously, and economically supported. Maintainability is a 
design parameter which reflects support considerations, and 
МЕТЕ Теса vestosimtegrated Aogisticsssupport. |8, р. 1471. 

Quantitative На В. values must be determined Бу 
Considering trade-off's with reliability and cost. Figure 
2-3a represents trade-off alternatives between reliability 
and maintainability at a constant availability level. 

Figure 2 - 3b shows the effect upon reliability and main- 
tainability of cóst and Figure 2-3c depicts the trađe-offs 
between availability, reliability, maintainability and cost. 


During this phase the system characteristics must be determined 
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COST-AVAILABILITY TRADE-OFF CURVES 
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апа used as guidance. The basic methods of construction 
(extent of modularity, accessibility, fastenings, plug-in 

and wire-in features, etc.), the location and circuit protec- 
tion of test points, the level of the lowest replaceable unit 
(LRU's), the extent to which automatic fault detection (AFD) 
and built-in test equipment (BITE) will be provided, the type 
and extent of fault isolation and localization software 
(programs for AFD and technical manuals and data for main- 
tenance men), and the nature and degree to which on-line 
performance monitoring and testing is to be provided,must all 
be specified during the Contract Definition phase. [4315 б, 39 0s 

Various models have been developed based upon mathematical 
distribution models to aid in quantifying maintainability 
2 ape 201] Иа, р. 137) [20, p. 3]. 

If we are to profit from the ground work Ете during 
the previous phases, the Acquisition and Operational phases 
must be guided by two paramount considerations. First we 
must be prepared to perform the management function of con- 
Pro contro TTM is the management function of making sure 
that plans succeed. In other words, it is the measuring and 
correcting of activities to ensure that these activities are 
comer ло сие во Бре расплемешеп ле Ер аппеа goals. T23, p- 507]. 
Buringzche Conceptii Formulation and Contract Definition phases 
standards for maintainability were established; during Acqui- 
Sition and Operations maintainability performance must be 


measured and deviations from the standards corrected. 
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Measurement is the second paramount consideration. Since 
we are concerned with systems availability which we plan to 
enhance through systems maintainability, we must have a means 
to test the systems maintainability to be able to quantify 
. the level of performance. 

Test procedures which test the entire system and not 


components of it, are very hard to find, and are the subject 


of this paper. 
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ЖЕ MODES OF FAILURE 


This chapter will attempt to analyze failure; its causes, 


pos, occurrence and effects. 


КОШ CHASSTEICATIONS OF FAILURE 

Failures can be classified according to location or cause. 
P-clussfrcation of de£íects according to the location of 
failure determines a weak point in a system and is the first 
Step needed to strengthen it. The reason for the occurrence 
of the failure can be analyzed as follows: 

Design defects: Defects attributable to design imper- 
fections. Pushing the state of the art or failure to consider 
all possible stimuli and responses of a system lead to design 
defects. 

Manufacturing defects:  Failures of this class occur as 
a consequence of violation of the technological manufacturing 
procedure chosen for the system or unit. The quality of the 
individual units and components of a system have unavoidable 
random variations. These manufacturing variations, decrease 
the reliability of some of the items or units. 

Improper use: For every system, restrictions are made 
on the conditions of its use. Rules are given for maintenance 
and operation of the system and its parts. Violations of these 
rules or restrictions may lead to premature failure. 

Deterioration (aging): No matter how high the quality of 


the unit and/or the system as a whole, gradual wear or aging 
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is inevitable. During the course of use and storage, іг- 
reversible changes take place in metals, plastics and other 
basic materials. The cumulative effect of these changes 
deteriorates the strength, coordination and interaction of 
the parts. This з may not be as readily apparent 
as an outright failure. The piece of equipment or system is 
Still operating but at a sub-optimal level of performance. 
All components, subsystems and systems are subject to dete- 


rioration which in the final analysis may cause failure. 


Pee LY RES OF FALLURE 
Instantaneous failure: Failures occur at the instant an 
operational parameter 1S exceeded. These failures are random. 
Typically the failure of the unit or system occurs independent - 
ly of how long it has been used or what condition it is in. 
An example of this, is connecting a 110 volt appliance to a 
440 volt power supply. The failure is instantaneous and 
usually readily apparent, no test may be necessary. 
Degradation failure: This failure occurs as"the result 
of the gradual change in operating parameters. Admissible 
limits for the operating parameters of a unit or system are 
established when it is designed. A performance parameter 
operating outside these limits is classified as a failure. 
ОЛО СБ], All unies vor systems aneysubject to wear or aging 
andmetheretone their condition is continually changing or 
transient; usually it is deteriorating. A minimum performance 
limit is established and the unit or system will approach this 


limit throughout its lifetime. This state is known as 
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“graceful degradation." A test procedure is needed to 
determine how much degradation has occurred. 

Interaction failure: A Situation in which several causes 
of failure act simultaneously. It is the most typical type 
of failure in practice. A vacuum tube may fail because it 
was Subjected to a random "peak" load, after graceful de- 
gradation had lowered its upper level limit. Тһе result is 
instantaneous failure. Interaction failure can be reduced by 


calibration which restores the upper limit to its proper 


level, if the amount of degradation is known. 


Seo CCURRENCE OF FAILURE 

The occurrence of failure is examined by analyzing the 
distribution of failures with respect to time, especially for 
components utilizing electronic technology. Figure 3-1 shows 
the classical failure rate for a finite population of newly 
manufactured components under a uniform operating stress. 
The failure rate curve is referred to as the bathtub curve. 
It is based on the presumption that failures will be fixed 
as fast as they occur, to keep the component operating. 
Virtually all components start their life with a very high 
initial failure rate, largely because of design and manufac- 
turing deficiencies which economical quality control cannot 
find. This period is known as a debugging, or burn-in or 
infant mortality period. Following the burn-in period, the 
component enters its useful life period. Toward the end of 
the useful life, so many parts are deteriorating so fast that 


maintenance becomes uneconomical, the failure rate begins to 
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rise and the component enters its wear-out phase or end 
p Nate 125, р. 2]. 

The classical failure rate has been challenged with respect 
вое Validity rom the user s "pornt ot vrew as applied to 
Fleetrical components Most contracts call for a manufactur- 
er's burn-in period prior to acceptance. For the user the 
burn-in period is nearly non-existent. With integrated 
Na "rcuilts, printed circuits, chips, 'transuüstors, etc., most 
electrical components are obsolete and are replaced before 
they enter the wear-out phase so it does not apply. 

Examination of the useful Life period calls for following 
a subtle line of reasoning and the solving of a contradiction, 
which this paper will not attempt. The first argument says 
that, given adequate preventative maintenance, experience 
has shown that the failure rate is fairly constant during 
this period. The second argument says that since the failure 
rate during the useful life period is determined by random 
failures preventative maintenance will have no effect upon 
the failure rate and therefore serves no purpose. Both 
anguments accept Ет farlure and the existence of 
degradation which may impede a component's performance while 


not lowering it to a "failure" status. 


Е. EFFECTS OF FAILURE 
The effects of failure can be as varied as the user, the 
component or the system which has failed. The effect could 


весе тош чот 1 сапое®Ео catastrophy and loss of life. 


2 





Е рег оноУФоианепеюо вЕССЕбО Ов failure, missions are 
curtailed or altered, duplicate systems (redundancy) are 


employed and several more expensive solutions are implemented. 
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Deterioration 


Random Failures 


Useful Life 





0981 OMTIRI 


"Bathtub Curve" of Lifo Failure Rate 
Adapted from Winlund (25 - p.5) 
Figure 3 - 1 
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INE RESBNT TEST PROCEDURES 


Before examining test Se, equipment levels will 
be defined. | 

Component: A component, (subassembly, assembly, unit ог 
group) is any collection of pieces which composes a sub- 
division of a set, piece of equipment or system. By itself 
it is capable of independent operation but it is not capable 
of performing a complete operational function. It is usually 
replaceable as a whole, but has parts which may be individu- 
ally replaceable. (Examples: Electronic power supply, radio 
receiver, antenna group, oscillator). 

Equipment’ (set): A component or components and necessary 
parts connected or associated together to perform an opera- 
Fronal#=iunction. (Examples: Radio receiving set, sound 
measuring Set, ren includes such parts and components as 
cable, microphone and meaSuring instruments; radar homing 
set). 

Subsystem: A subsystem is defined as a combination of 
equipments, components, etc., which perform an operational 
function within a system. 

Subsystems from the major subdivisions of systems. 
Examples: (a radar station, fire control subsystem, a radio 
татти uM ест Ил таи 

SST ОЛ LIectroniesy.: wi combination offequips 
ment and/or subassemblies, generally physically separated 


when in operation and other such components necessary to 
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perform an operational function or functions. Examples: 
(Antiaircraft defense system including tracking radar, computer 
and gun mount; GCA electronic system; communications system: 

A link including transmitter, medium of transmission and 
receiver). ле рат а 

In order to determine the level of operation of a test 
procedure we must specify the level of operation of the item 
being tested. The composition and the characteristics which 
must be measured at each level must be specified. Once the 
characteristics being tested have been defined, the level of 
operation of the item and the test system can be defined. 
Table (4-1) attempts to categorize and arrange the equipment 
levels and the characteristics measured. 

A capsule summary of the prominent test procedures used 
in Naval Communications will follow. It must be understood 
that variations in tests occur from ship to ship and fleet to 
fleet, however an attempt will be made to provide a basic, 
objective guideline and discussion of each procedure. 

The Quality Monitoring System (QMS) is a subsystem level 
performance monitoring procedure. It is composed of hard 
wired, monitoring equipment which is interfaced into the 
communications system at key points. It can be used on-line 
or off-line to measure degradation of selected character- 
istics. It requires extensive training and is expensive to 
install, however, quantum improvements in communications 
system performance can be obtained when QMS is properly used. 
OMS is capable of performing qualitative, voice, systems 


level testing. [27] 
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Pomsee is a subsystem level calibration test. It can be 
performed aboard ship but its best application is ashore at 
a depot level maintenance facility. Equipment must be removed 
and dismantled and is tested on a test bench. It is costly, 
time consuming and requires a high level of training to be 
properly performed. It is best applied to pieces of equipment 
which are known to have failed and not as preventative main- 
tenance or in order to locate a failure. 

Built-in test equipment (BITE) is a confusing title to 
apply to a test procedure. QMS could be considered built-in 
test equipment, however the term usually applies to component 
or equipment level "self-test" procedures. BITE is capable 
of being applied at the subsystem or system level, however, 
it is not done in practice because of cost. It usually pro- 
vides a GO/NO-GO indication, a continuity check of cables 
and connections and requires an insignificant amount of time 
гобрек окт. No special training is required to use the test. 
Its primary draw-backs are cost, one test circuit or piece 
of test equipment is required for each item being tested, 
and the sparcity of elle sheGeingeimrormatlion ayalllable7 
ШЕ usually won't say what component failed or why. [28, p. 
4-1] 

The Standard Measurement Technique (SMT) is a quantitative, 
system level, test which measures the flow of information 
being processed by the system. It can be hard wired or 
patched in. It is an inplace, off-line test which is capable 


of quantifying the system ability to process degraded inputs. 
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It requires very little training, is moderately expensive 

and can be performed in minutes. SMT is being developed by 
the Naval Electronic System Command and a complete description 
of the system is provided in the following chapter. 

The Planned Maintenance System (PMS) is a subsystem level, 
off-line GO/NO-GO test. It can be used on a daily basis and 
requires B-school training to perform. Portions of it can be 
performed with less training and portions will require higher 
preeruriMg mMOrestime and a test bench. PMS can be hard wired 
into the system but it is usually patched in, using portable 
equipment. The use of portable test equipment presents a 
problem because the equipment is often "banged" out of cali- 
bration rendering it unreliable. PMS is used to locate a 
failed piece of equipment. The 3M cards are used with PMS. 

The 3M test procedure is a subsystem level maintenance pro- 
cedure. Basically it is a check list, set of instructions 
listed on a card which describes a test, lists the equipment 
necessary and gives step by step instructions on how to perform 
the test. It can be performed off-line, or on a test bench. 
Fleet studies have indicated that component and equipment level 
3M/PMS testing is performed in the fleet but that subsystem 
level testing is not performed. [29] 

Table 4-2 presents the various test procedures and attempts 
to show the significant criteria which can be used to compare 
them with each other. | 

Тһе definition of each criteria used in Table 4-2 is as 


follows: 
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Component, equipment, subsystem and system - as presented 
at the beginning of this section or Table 4-1. 

Detect failure - can the test procedure detect a failure 
at the highest level of performance given? 

Detect degradation - can the test procedure detect de- 
gradation at the highest level of performance given? 

Quantitative - does the test procedure provide a quanti- 
tative read-out (as opposed to a GO/NO-GO type output) at the 
highest level of performance given? 

Measure flow of information - does the test procedure 
have the ability to measure the flow of information in the 
system? 

Performed daily - is the time needed to perform the test 
such that it can be performed daily? 

Remove and replace - does the item to be tested have to 
be removed in order to be tested? 

НОО Во За igh level of training (B-school) 
necessary in order to be able to perform the test? 

It must be pointed out that table (4-2) does not consider 
em COS пони аси — procedure. Cost analysis must be con- 
sidered before any test can be considered optimal. 

The items TES should be given major consideration are 
System, measure flow of information, performed daily and high 
level of training. These are the items which the user of the 
test equipment is interested in and upon which his judgment 


Сесе ое сисе иперт 20 use thg test procedure, 
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V. STANDARD MEASUREMENT TECHNIQUE 


A. GENERAL CONCEPT OF STANDARD MEASUREMENT TECHNIQUE 

Studies and communication exercises performed over the 
years have pointed out again and again, the fact that an 
adequate method of determining communications SYSTEM perfor- 
mance does not exist. This chapter will attempt to offer a 
ЕС А Соло this problem. 

Performance evaluation has been confined to a lengthy 
and complicated (POMSEE etc.) evaluation of individual equip- 
ments and proceeded on an assumption that proper operation of 
system components ensured satisfactory operation of the total 
system. This viewpoint ignores the fact that systems are 
often synergistic and presupposes component performance levels 
are linear and arithmetically denke. 

The emphasis on individual equipment performance often 
resulted in catastrophic equipment failure as the sole criteria 
for maintenance action while quantum system performance de- 
gradations went largely unnoticed. A Е 

panajoraweaknessmosEthe couventyonalt methods Hor testing 
and maintenance of communications systems is the inability to 
test for varying levels of degradation in the system. Attempts 
Best abi Tvshz23zeommüunicatiens Circuit using ostensibly opera- 
tional equipments often resulted in failure. Diagnosis of 
cui malrtunctions did not isolate faults to tthesship or 
сито он се сл пе око ен спогршеке ОП Іһе ісепізізсастоп of 


faulty system components at the specified site. 
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The genesis of some of these problems can be found early 
in the equipment life cycle. During acquisition equipments 
are tested by various parameters, levels, procedures and 
conditions. The ambiguities in equipment performance levels 
often limit the amount of predictive analysis a communications 
systems engineer can perform prior to system implementation 
resulting in performance requirements written post facto. 

Subsequent system testing (OPCAP, INSURV, etc.) frequently 
utilize a shore termination as an operational standard. Since 
the standard has not been tied to a reference point utilizing 
strict methodology the predictive performance aspects of the 
system operating with different terminations often suffer. 

A need for formalized testing methodology, from equipment 
level to systems level, is evident. An additional necessity 
is the establishment of minimum system performance levels. 

All too often system levels are established at or near design 
levels (as new or refurbished equipments) without an adequate 
determination of allowable performance deterioration due to 
aging. As a consequence, equipment replacement or overhaul 
can become a qualitative judgment without adequate consider- 
ation of an equipments effect оп 185 ог запой 11 агу 
equipment. 

The Standard Measurement Techniques (SMT) will fill 
this long-standing need. The SMT will provide ships and shore 
Stations with techniques and procedures for testing and 
measuring overall communications effectiveness on a system, 


subsystem, equipment, and circuit level without an emphasis 
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оп automation, the need for general purpose digital com- 

puters, or highly trained operators and technicians. With 

the use of magnetic tape inputs and the traditional approach 

to communication testing, a valuable new technique is developed. 

The evaluation of system operation can be determined rap- 
idly by the following technique: Prerecorded teletype signals 
with various levels of degradation, such as distortion and 
atmospheric disturbance will be available on magnetic tapes. 
The tapes allow a communication system to be checked under 
varying conditions that are experienced in actual ship and 
shore operations. 

The National Bureau of Standards and the Pacific Missile 
Range will pretest, verify and supply the magnetic tapes. 

The various test signals and the utilization of existing 
displays will permit interpretation on, a GO/NO-GO basis, the 
ability of the system to meet functional specifications. 

To provide for universal testing it is necessary to 
accomplish the following long range goals: 

а. “Пегіпе апа describespxresent systems. 

An Engineering Data Register (EDR) has been promul- 
‘gated сае с пе” ретелпепснератастек ив висв ата 
specified parameter values for each Naval Communica- 
tions System. 

b. Establish numerical systems performance levels. 

Reference Level Measurement Guides (RLMG) will 
establish acceptance levels criteria for systems 


AARE pment They will Specify the following 
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for a particular communications system: 

- which parameter is to be tested. 

- the value or level for measured parameter. 

- which measurement method in the SMT should be used. 

C. Develop techniques for measuring system performance 

levels. The SMT defines the applicable technique 
for measuring each characteristic in conjunction 
with testing the overall system performance. 

In a letter to the Chief of Naval Operation (CNO), dated 
21 Sept. 1972, the Commander, Naval Electronic Systems Command 
Stated the following: 

"The purpose of SMT is two-fold. First, it measures 
communications system's performance to determine system's 
operational capability. Secondly, it has the capability to 
locate or identify equipments that are not performing properly. 
The tests are conducted on record, and voice communications 
systems under simulated, controlled, repeatable conditions 
Piet exerciserail parameters оя а system. The сезе сап ре 
Pentormcdgat Seagor sin port without need fior assalstance from 
other ships or shore stations." [31]. 

In ose to contrast the SMT approach with present testing 
methods, the following distinctions are made: 

Philosophy of present testing methods: The basic principle 
of the current testing methods is that the better a system is, 
Еве less it will distort the mosit perfect signal. Tests are 


ponductetde,ser distoruron offa single characteristic. 
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Philosophy of SMT testing methods: The basic principle 
of testing methods with SMT is that the better a system is, 
the greater degradation of input it will withstand and con- 
tinue to operate. Tests are conducted for system response 
to maximum distortions of multiple characteristics. [32] 

OX types of system degradation are examined with SMT: 

. Teletype Distortion: Teletype systems are tested by 
introducing five lines of test in successively increased 
amounts of distortion from a tape, each line containing a 
different type of distortion. The terminal teletype of the 
subsystem is then checked to determine at what point the 
system fails to produce an error-free copy. 

. Atmospheric Distortion: Teletype systems are tested 
by introducing ninety lines of test with Multipath, Noise and 
Fade from a tape to check the system's diversity capability 
and the receiver subsystem’s response to signal amplitude 
fluctuation and sensitivity (S/N). The terminal teletype of 
the system/subsystem under test is examined to determine the 
system's capability to produce an acceptable copy. 

и Ггапст е еглересетай S Y SICe ns arene neers 
15 gr by examining its output with a spectrum analyzer 
салар сетсаасиотЕ тоацта слом harmonic thstortron, inter- 
modulation distortion, and suppressed carrier level. 

. Articulation Index (AI): Voice systems are tested by 
inputing various audio frequencies and a precise amount of 
noise from a tape. The receiver output is then measured 
with a noise meaSuring set to determine the intelligibility 


capability of the system. 
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. Multicoupler/Coupler Swept Frequency Response: A 
system's multicoupler is tested by introducing various 
frequencies from a tracking generator into the system's multi- 
coupler and examining its output with a spectrum analyzer to 
determine loss or gain, bandpass, ripple factor and skirt 
characteristics. 

. Antenna Swept Frequency Response: А system's antenna 
is tested by introducing a continuous spectrum sweep from a 
tracking generator at the antenna through a directional 
bridge. The output is then examined on a spectrum analyzer 
to determine the standing wave ratio and resonant frequency 
of the antenna. The antenna signature presented on the 
spectrum analyzer is photographed and maintained as a main- 


tenance standard. (827 


СРЕСТЕТПС OBJECTIVES OF SMT 

We are primarily concerned with the questions of whether 
communication equipment is operational, and at what degradation 
level it becomes inoperable. Therefore, the scope of the 
Standard Measurement Techniques program involves four levels 
of testing, each progressively more detailed. Tests are: 

IO o LEMA TES 

De 05 SYSTEM TEST 

SEE РОЈТРМЕМТЕ ТЕО 

ПРИ | КСЈ TEST 

Тһе tests are designed with the following objectives: 

а lest toresystem response to maximum distortion of a 


ПОЕ ИЕ т спарасвектсеісе үлеп а minimum ot tests. 
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| 
| 


Reliability; tests are to duplicate real world 
communications environments and will measure system 
performance under worst-case conditions. Systems 
orientation; emphasis on providing realistic simulated 
inputs to systems and observation of resultant outputs. 


Define numerical system parameters for inclusion in a 


Reference Level Measurement Guide (RLMG) and establish 


acceptance/rejection criteria. Performance level 
indices; numerical analysis of system performance vice 
a reliance on GO/NO GO judgments to provide a measure- 
ment of system quality. 

Utilize the existing system equipments to interpret 
results. Economy; an avoidance of automation, com- 
puters, and highly trained personnel. Design goal is 
eo ром се оса! омтисараба/тсу ELOY Mess than 55000.00 
per site. 

Minimize the number, sophistication and time of tests 
and technicians. Speed; complete systems tests in 
less than 15 minutes рек гез етпрісселіу) сезі 
procedures that are straightforward and easily under- 
stood by operational personnel at the fleet level. 
Maximize confidence in the validity of system acceptance. 
Establish capability to rapidly and confidently as- 
certain particular system, subsystem or equipment 
characteristics which do meet specified parameter 


va mies or tolerances: 
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Е. Commonality for all communications systems. Inde- 
pendence; performance of SMT measurements that are RF 
silent, self contained, and can be accomplished with- 
out support of off-site activities. Devise and 
establish Standard Measurement Techniques (SMT) for 
testing and measuring system performance levels of 
communications systems employed at communications 
Stations afloat and ashore. Develop and prepare 
Detailed Test Plans (DTP) tailored to a specific 
platform/site and establish procedures for reporting 


results of testing performed. [2:359] 22.1 


NE TEST PROCEDURES 
The RLMG describes system tests necessary to ascertain 

that equipment characteristics meet the specifications listed 

in the EDR. 
Test Notes 

^a. Each test procedure is furnished with a block diagram 
for use during test set-up. The test equipment is 
shaded for clarity. 

Б, спе hour — period is recommended to stabilize 
the system and test equipment. 

с. Safety Procedures: NAVSHBPS 250-660-742, Electric 
Shock Its Causes and Prevention, should be made 
available to personnel engaged in testing. 

d. Equipment that is essentially passive, e.g., receive 


and transmit switchboards and interconnecting lines, 
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is assumed to have been tested prior to start of 
system tests. [up pl. 
Functional System Tests (Figure 5-1) can be used for 
systems orientation. 
System Test Part 1. TTY end equipment. 
System Test Part 2. System Test with TTY Distortion 
System Test Part 3. Receive System Test with Atmospheric 
Distortion 


System Test Part 4. Antenna (Transmit and Receive) 


System Test Part 1: TTY end equipment 
System Test No. 1 15 a teletype "end equipment" functional 

check of reperforator and page printer only on a closed loop 
basis via the DC patch panels. Teletype distortion tape: 

Five lines of test, each line containing a different type of 
aS Cer, fon enc lle MARK BIAS, line 2 SPACE BIAS, line 3 
SWITCHED BIAS, line 4 SPACE END, and line 5 MARK END will be 
transmitted in successively increased amounts of distortion, 

% increments, through 45% distortion. The end equipment or 
teletype of the system/sub-system under test will be checked 
to determine at what point the teletype machine fails to 
produce am “error-free copy. ИЗ. р т 

Procedure details for system test part 1: 

dE CCC Ne rape recorder rtostne DC Patch panell Patch 
Bee patel anell to page printer: 

b. Send type test message; TEST THE QUICK BROWN FOX JUMPS 


OVER THE LAZY DOG'S BACK 1234567890 testing. 


c. Verify printed test message at page printer. 
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а. Repeat test message for each channel of "Р" system. 

e. Objective of the test is to ensure proper operation 
Gr perforators and page printers. Figure 5-2 is a block 
diagram of the test. Table 5 - 1 is an example of the 
printed page messages. 
System Test Part 2: System test with TTY distortion 

System Test Part 2 is a quantitative semi-full system 
loop test. The following technique will demonstrate the over- 
all system performance dnd its Capability to perform with 
known distorted signals. Input to the system is provided by 
a magnetic tape recorder. The tape recorder will have four 
fay) anstrumentation channels. Three instrumentations channels 
will have prerecorded teletype signals which are degraded at 
successive intervals with various levels and types of dis- 
tortion such as space bias distortion and spacing end 
distortion. Тһе fourth instrumentation challel will be used 
for the tape recorder speed servo control loop. The test tape 
used in test Part l is used for this test. [E р E 
Procedure details for system test Part 2. 

а, the tape recorder 1S patched via an interface unit, 
to the DC patch panel. The transmit antenna lead is removed 
from the antenna and patched to a dummy load. The transmitter 
is patched to the monitor equipments. The receive antenna 
lead is removed from the antenna and patched to the monitor 
equipment. This series of connections makes a loop permitting 
a test signal to be introduced by the tape via the DC patch 


panel. The test signal 1S sent through the transmit system 


47 








р. С. 


5 
3 
ра 
515 
O 
E 
<Q 
As 


TAPE DECK 


Figure 5 - 2 TTY Test 


System Test Part 1 "P" System 
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БИЗ ОР TEST 





to the monitor equipment, to the receive system to the 
page printer. 

b. Send tape test message TEST THE QUICK BROWN FOX JUMPS 
OVER THE LAZY DOG'S BACK 1234567899 TESTING. 

с. Verify printed test message at the page printer. 
When a NO-GO condition appears maintenance work can be 
accomplished by use of the Sub-System test 1, after all 
systems tests are complete. 

d. Repeat message for each channel of "P" system or by 
using an interface panel test all channels at one time. 

e. The objective of this test is to ensure proper 
operation, compatibility and performance of the system, ex- 
cluding the two antenna systems, the effects of the propa- 
gation medium and the transmit TTY equipment. A block dia- 
рам, #19. 5-3, is supplied for guidance, and table 5-2 
lists the equipment necessary to perform the test. 

System Test Part 3: Receive System test with atmospheric 
EuGtortion. 

System Test Part 3 is a quantitative, receiving system 
ЕСЕ Те ооу па teehnique will directly imply the level 
ої performance to which the system will operate. Input to 
the system is provided by the magnetic tape recorder as used 
ma Test No. I, Three) instrumentation channels will have pre- 
recorded multichannel RATT messages which are degraded at 
successive intervals with various levels and types ої atmos- 
Ppherrcocondrtzons such as time spread distortion and noise. 


The fourth instrumentation channel will be for the tape 
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ШТ ТОВ TEST РОШПЪМЕМТ POR SYSTEM TEST PART 2 
IE р. 2 а 8] 


poscrrpEron 
1. Magnetic Tape Deck 


Pee interface Unit 

Be Spectrum Analyzer 

4. Frequency Counter/Generator 

Ew Attenuator 1205 а 

Ки Attenuator 12 db 

M Micro Voltmeter 

8. Dummy Load 

9. Power Dividers (2) 
10. Line Fuse for Spectrum Analyzer 
li. Test Tapes 


Ip Patengcords 


table (5-2) 
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Figure 5-3 Semi-Full Loop Test 


System Test Part 2."P" System 
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recorder speed servo control loop. Atmospheric Distortion 
есре-ниипевув  "пезної test, including thirty lines of MULTI- 
PATH distortion to determine the system diversity capability. 
Thirty lines of FADE to exercise the receive systems capability 
to respond to signal amplitude fluctuations (AGC). Thirty 
lines of NOISE, to test the receive systems sensitivity (S/N). 
In all cases, the end equipment, teletype, of the system/sub- 
system under test will be checked to determine the systems 
Барара 1 е Еойркоаисе an acceptable copy Sl error in ten 
lines acceptable, for the system under test.) [50 11] 
Procedure details for system test part 3: 

а истини -несштен то utilized to Supply an RF input to a 
Шеп Јо помор со ма Сб ehe Kmput from the tape recorder. 
The test signal is processed through the receive system (in- 
put is below the antenna) to the page printer. 

руши о = паше соксеа ње Signals, With atmospheric distortion. 

c. Verify printed test message at the page printer. 

When a NO-GO condition appears maintenance work can be 
accomplished by use of the Sub-System 2, after all systems 
tests are completed. 

а. Repeat test signal for each channel of "P" system or 
by Tont: an interface panel test all channels at the same time. 

а ит лешеве своје eL this Есес tofdemonstrate the 
operability and performance of the receiving system when 
exposed to the varying effects introduced by a test signal 
Eh koown quantıt es -ofgatmospheric distortiom. A block 
Иш лса ЖОО сл 1Sesupplled for guidancefand' table 5-3 15 


a list of equipment needed to perform the test. 
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ist Ors TEST ВООТЕМЕМТ FOR SYSTEMS TEST PART 3 
(94, р. 2 - l5] 


Description 
l. Magnetic Tape Deck 


2. Test Tapes 

3. Synthesizer 

4. Mixer 

ККК Attenuator 120 db 
6. Attenuator 12 ар 
I Micro voltmeter 
8. Power Divider 

9. Line Fuse 


ШІ Patch cards 


Table (5 - 3) 
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Systems test Part 4: ЕН and Receive Antennas 

System Test Part 4 is a transmit and receive antenna 
test. Input to the system is provided by the magnetic tape 
recorder. An instrumentation channel will have a prerecorded 
two-tone Signal. This two-tone Signal is aioe tied to the 
transmitter with a full power setting utilized for the trans- 
mit antenna. The net and receive antenna signals are 
evaluated at the Current Monitoring Panel (CMP). The receive 
antenna signal is provided by the transmit loop of the "P" 
System at a minimum level to prevent multicoupler overloading. 
Ш, Р. 15] 

Procedure details for system test Part 4. 

a Patch tape recorder to the transmitter Switchboard. 
Merach wae Current сол: to transmitter antenna lead 
and to the receive antenna lead. Connect both current 
coils to a current monitor panel. Patch a spectrum 
analyzer to the current monitor panel. 

b. Record spectrum analyzer display and determine the 
standing wave ratio and resonant frequency of the 

щапшсешша 

с. Send taped two-toned signal. 

А. Repeat test for each antenna. 

e. The objective of this test is to determine the 
efficient and inefficient frequencies (or frequency 
ranges) for each antenna. Reference Fig. (5-5) for 
аА лоск ачасгап апа сав 1с (5 4) for la list of equip- 


ment necessary to perform the test. 
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ПРА RORE TESTE EOUTPMENTTFOR"TSYSTEMS TEST 4 

ИЗД oe 2%®— 211 

Description 

1. Magnetic Dane deck 

2. Test tapes 

EN Spectrum Analyzer 

4. Antenna current monitoring panel 

Bee) Current coils 


6. Patch cords 


Тар ке 07 
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Sub-System Tests (Second level of the SMT) 

The Sub-System Tests are sec ones tei solate a particular 
part of the communication system when a NO-GO condition 
appears during System Tests No. 2 or No. 3. There are two 
tests, two parts to each test. By a process of elimination, 
each test will pinpoint the NO-GO equipment or equipments 
for that particular test. 

The other components in the circuits are essentially 
passive, i.e., receiver and transmit switchboards and inter- 
connecting lines. Existing standard tests can be utilized 
for these items. The foregoing technique isolates the 
transmit and/or receive section of the overall system to еп- 
able the test personnel to pinpoint the exact NO-GO equipment. 
IRE t8] 

Procedures for Sub-System Test No. 1 
а. Set up test and monitor equipment as in Systems test 

Part 2 except that the tape recorder is patched to 
the transmitter switchboard instead of the D. C. 
patch panel. The tape recorder sends the test 

"signals "E апт Е section or the, Ppi 
system. After the transmitter is tested and verified 
the test tape input reverts back to the multichannel 

. keys and the test is run again. Transmitter Spectral 
Purity: The transmitter under test will be tone 
modulated, 8 or 16 tones, as required and the output 


Will be examined on a spectrum analyzer to determine 
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the spectral purity, (% of modulation, harmonic 
distortion, intermodulation distortion, and suppressed 
Carrien level) that applies to the transmitter undër 
test. | 

b. A black diagram, Fig. (5-6) is provided and table 
(5-5) is a list of equipment necessary to perform 
the test. 

Procedures for Sub-System test92: 

И ЛС е аре кесовас 150) сте receiver bo al 
and test the multichannel convertor. After the 
convertor is tested test the receiver as in System 
че ба ©. 

Ь Шосе diagram, Fig. 15-7) is provided and table 
(5-6) is a list of equipment necessary to perform 
the test. 

Voice Communications System Test 

The Voice ystem Test is a test procedure whereby a 
Systems ability to transfer intelligible information is 
шаси сив гесетепсен о апжаттасыЛастоп Index (AI). “АТ 
has been shown to be a function of the link description (type 
and frequency) , апа һе ые ус н е Бие ”аеттуес апсі іпсеутТтету- 
пата ист спао Бе and I; respectively; heice for anyi given 
communieations systems of such configuration that link and 
interferer descriptions and frequencies are known. AI can 
be established as a function of SRF and I." би 
Values of AI have been determined in terms of audio S/N 


kalues permit Elng volilce System S intelligibility to be 
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НЕШЕ ПЕ ЛОВ МЕ МИ POR SUB-SYSTEM TEST 1 
31, р. 3 — 4] 

Description 

1. Magnetic Tape Deck 

2. Frequency Counter/Generator 

3. Spectrum Analyzer 

4. Attenuator 120 db 

5. Attenuator 12 db 

SENE MDEScroSvoltmeter 

7. Dummy Load 

8. Power Divider 

9. Test Tapes 


10. Patch Cords 


Table (5-5) 
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Sub-System Test 2 "P" System 
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ІШЕт ӨК ТЕСП ЕОПІРМЕМТ FOR SUB-SYSTEM TEST 2 
315; р. 3 = 9 | 


Description 
1. Magnetic Tape Deck 


Ao  dWsXeke Шарев 


Table (5 - 6) 
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measured in terms of a S/N ratio. | / НА о ісщіаєтом 

Index (AI): A tape containing various audio frequencies and 
a precise amount of noise will be introduced into the system 
and the receiver output will be measured with a noise 
measuring set to determine the intelligibility capability 

of the system. This test 1S only used in voice systems. 

fee pe 111). 

Procedure details for voice system test: 

а. A taped signal containing audio frequencies and 
noise is processed through the receiver and the 
output signal is measured by a noise measuring set. 
The noise measuring set looks at the noise passed by 
8 1 Гете іссасесй an the voice band width. The 
difference between the input and output noise levels, 
expressed as S/N is equated to AI which is equated to 
еее тот Су. 

b. This test procedure is used to determine the perfor- 
mance of a receiver configured for voice communications 
only: 

са еа сшке 5—8 provides с OE diagram of the test and 
table 5-7 provides a list of equipment necessary to 


perform the test. 


В. APPLICATIONS AND RESULTS 
Demonstrations have been conducted on board fleet units 
and at shore activities to validate design decisions made 


during SMT development, examine the universality of the 
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VOLCE COMMUNICATIONS TEST EQUIPMENT. 


Description 


Random Noise Generator 

True RMS Voltmeter 

Amplifier 

Wave Analyzer 

Voice Interference Analysis System 
ај Бал“ Voltmeter 

Таре Recorder 

Noise Measuring Set 

Synthesizer 

Attenuator 


Attenuator/Rotary 


Table 5 - 7 
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procedures, and to introduce communications system users to 
the capabilities of SMT. Demonstrations have been conducted 
at or on board USS England, USS Preble, USS псе рај 
NAVCOMMSTA SFran, Naval Postgraduate School, Monterey; USS 
Constellation and other Commands. 

SMT performance has demonstrated that the original Long 
Range Goals can be satisfied. A number of present systems 
have been defined and described in the EDR. Numerical sys- 
tems performance levels have been ее веса Ее ЕИБ 
and SMT has shown the techniques for measuring the systems 
performance levels. Several specific SME objectives Nave 
also been satisfied. SMT tests are straightforward and can 
be understood by operating personnel at the fleet level 
after a short, formal course at Radiomen "B" School. То а: 
large extent, existing test equipment can be utl 11 еа 
even MHtests permit the entire communications system to be 
examined. Test sophistication is placed in the test equip- 
ment, not in 1ts+operation or in the technicians. SMT test 
sequence can be applied to any communications system, afloat 
or ashore, unaided. | 

During December, 1972 the USS Constellation (CVA-64) was 
used for a demonstration and some of the test results are 
presented here. All communication systems available for 
operation were tested. Several significant problems un- 
covered were corrected or brought to the operator's attention. 


Noteworthy are the facts that: 
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1. The SMT disclosed severe maladjustments and failures 

not discernible by other test methods and 

Pernice eame taken CO accomplish the tests was con- 

servatively less than five percent of that required 
by more conventional means available to the ship 
personnel. 

Problems encountered which could be corrected by adjust- 
ments and/or alignments were corrected. Problems requiring 
maintenance or repair were explained to cognizant, responsible 
m 

It must be pointed out that the test sequence permitted 
testing of parameters not testable by other methods and that 
the results were able to point out pieces of equipment which 
were functioning but operating out of specifications. |38) 

Table 5-8 ds an example of the SMI TTY test. After the 
page printers were checked, the alternate channel convertors 
for the "P" system (UCC-1 channels 1 to 16) were tested and 
6 out of 8 were found in need of maintenance. All "n" 
system channels (low. tones) were adjusted and their perior 
mance was greatly improved after the SMT tests pointed out 
weaknesses. A.total of 23 transmit and receive multicouplers 
were tested. Ten of the 23 were found to be out of specifi- 
cations. Twenty-one HF voice receivers were tested, operating 
in upper side band (USB), and 4 were found in need of main- 
tenance. Twenty-one UHF voice receivers were tested and 12 
"fair" or "poor" performing units were found. Eighteen UHF 


transmitters were tested and 6 were operating within 
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SMI Bry TEST 


ot tas ES | IT cand Pige Printers) were” per= 
formed 11 and 12 December 1972. Eight of the best operating 
machines were selected as reference standards for all other 
и сејсота сл cule tests. 

A) осе Noir Ted StOrtion were introduced to each 
machine (eight at a time) in 5% increments, until the stress 
point of each machine was reached. A machine is considered 
to be in excellent operational condition if it can accept 40% 
distortion and print clean copy; 35% distortion input with 
clean copy is considered good; 30% fair; 25% marginal and 20% 
or below to be operationally in poor condition. 

Results of these tests were: 


Max. Distortion Max. Stress Type Distortion 
Machine with Clean Copy Point at Max. Stress Pty 
ELES KYBD 405 45% Space Bias 
РС ВЕСУ 35% 40% Space Bias 
LIPZBZERYBD 40% 45% Space Bias 
LTP-B RCV No test - Machine would not space 
LTP-PE KYBD 45% 45% l hit on Mark Balas 
LTP-N4 PP 40% 45% Space Bias 
LTP-PF KYBD 40$ 45$ Mark End 
ЕТР-М3 РР 35% 40% Mark End 
MTS EY B р 35% 40% Mark Bias 
LTP-A KYBD 30% 35% Mark Bias 
Вр КС 35% 40% Mark Bias 
LTE NIPP 40% 45% А11 
*LTP-PG KYBD 355 405 Space Bias 
ELTP-N2 PP 35$ 402 Mark End 
LTP-N5 PP 35% 40% Space Bias 
LTP-N7 PP 355 405 Space Bias 
ЕЛІР-ТСІ РР 30% 35% Mark Bias 
ШЕДІН- РЕШКЕУ 40% 45% Space Bias 
КИ Г- ТРОН УВО 35% 40% Mark End 
LTP-PC RCV 45%. Шит ор Space Ема 
LTP-PB KYBD 35$ 40$ Mark Bias 
Metro CH £8 | 25% 30% Mark Bias 
*NOTE 
РСК BON MarguüDNadjustmentSmRHegurredE- S characters short 
LTP-N2 PP Spacing and line feed problem 
LTP-PC KYBD Margin adjustment required - 2 characters short 

[38] 


Table (5 = 8) 
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specifications. Seven HF transmitters were tested and none 


were found to be operating within specifications. [38] 


E. PROBLEMS WITH PRESENT PROGRAM AND AREAS FOR FURTHER STUDY 
1. The SMT program has completed some system and sub- 
system feasibility studies and needs to devote additional 
attention to its final two levels of testing, i.e., equipment 
and circuits. 
2. Once laboratory procedures and instrumentation re- 
quirements have been established alternate, suitable, test 


equipment must be examined and a trade-off analysis conducted 


between equipment cost and equipment capability insorderzre 
reduce the price of each set of test equipment from $48,000 
to the stated goal of $5,000 per set, while still satisfying 
chesmissien ef SMT. The ideal cost effective solution, and 
one which certainly is feasible is to determine what pieces 


of test equipment, presently in the Naval inventory, coul 


а ре 


Dogg ес тен ом tose. did themeby Noti Incur any "addig 


спа еппіртепс expense mathe goal should bez that by 


— 


providing a trained radioman оце оре пише сем conduct the 
test at no additional expense to the Navy other than the cost 
of his EE and the tapes. y | 

Бг ере теге 25 tofbefrbeZultimatezuser of SMT, the 
fleet must have trained personnel available which implies a 
ТЕ and fre Beuctous, etc. These needs 


must be planned for as long lead time items so they can be 


available when SMT is vailable for fleet use. 
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А Опе Gu chesgeals of the SMT program is to replace 
several less efficient tests presently being used. These 
tests and/or procedures which are no longer needed must be 
identified and removed from the inventory. The SMT program 
should be able to pay for itself with tests, equipment and 
training it replaces while at the same time it enhances the 
tests which it complements. 

5. The SMT program must keep in mind its long range 
goals and specific objectives and continue to devote its 
attention in these areas. Work with the fleet should be 
secondary to these goals and objectives until the total 
package is prepared. By dividing its attention between pro- 
gram development and fleet applications it will most likely 
result in a less efficient product and it will extend the 
development phase longer than would be otherwise necessary. 
These detractions must be weighed against possible benefits 
of present fleet utilization. 

6. When Naval Electronic Systems Command has completed 
the SMT development program, it must have prepared an imple- 
mentation plan or OPORDER to introduce its technology to the 
fleet. ‘The ркавсетениогтавеспполоеосу,; Ley beget hect veland Tully 
utilized, must be prepared, programmed and conducted in a 
well planned and executed maneuver. 

7. Once SMT testing procedures for communications 
System, Sub-system, equipment and circuits have been designed, 
an adequate level of testing at each level must be developed. 


The depth, intensity and frequency of tests for components 
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and systems must be determined. The areas of degraded mode 
(total systems degradation or component failure which re- 
quires use of an alternate procedure) and systems sensitivity 
to atmospheric or operational phenomena must be determined. 

8. Studies must be conducted to determine the time period 

a calibrated (SMT tested) system remains in calibration. Іп 
order to ensure optimum systems availability should the 5у5- 
tem be tested daily, weekly, once a quarter, etc.? Once 
this "validity interval" is known an implementation program 
must be drawn up which would select either or both of the 
following courses of action: 

1) I£ the validity interval is determined to be measur- 
able in extended periods of time (month or more) the 
SMT test Sequence could be administered to a ship as 
part of its regular overhaul, pre-deployment ORI, or 
during every in-port period. This approachtwould 
dictate mobile teams stationed in key domestic and 
foreign ports. 

e a a a er va 15 measurab lesan hours}, days 
or weeks it may be desirable to have each major 
ата unit able to test itself on a frequent 
schedule. This approach would permit a ship to соп- 
tinuously monitor its communication system and achieve 
maximum availability. This solution would be expen- 
sive in terms of manpower, equipment and training. 

A trade-off analysis should be conducted between the 


resultant availability and cost. 
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It пау be determined that the optimum solution is a blend 
of the two approaches with major ships Carrying its own SMT 
equipment while mobile teams provide periodic readiness in- 
spections. Once either or both alternatives 15 selected, 
implementation plans must be раса ana funding sought to 
accomplish the established goal. 

9, Voice measuring systems must be refined. The present 
procedure calls for looking at three 100 Hz windows in the 
voice band from 200 to 6100 Hz and corre lating ene data to 
determine intelligibility. Studies must be continued to 
determine the optimum location of the three windows and to 
Feiinesthe procedure: 

10. Studies must be conducted to determine the signal 
reproduction loss due to multipath reception and the effect 
of quad diversity upon the паса 

11. Cross talk between systems and components must be 
studied so that its effects can be predicted, tested and 
Ponrocted- 

МУН Studies o£ compatibility of the SMT procedures and 
Secure systems must be conducted. Until SecureWwsystemsecan 
be а into communications systems test sequence the 
sequence will not be able to realize its full potential. 

13. тре validity of the atmospheric test must пре = 
creased. This can be accomplished by improving the atmo- 


spheric simulation techniques. 
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НОУ ПРЕ О АИ АЛ ЭЕМЕЕТТЬЮАКО АРРЕШСАТТОМ5 ОЕ SMT 


The potential applications and benefits to be derived 
from applications of the SMT concept are numerous. The ben- 
efits include increased standardization, more definitive 
specifications for procurement, reduction of communication 
systems duplication, reduced manpower needed to support 
communications, increased availability and a means to mea- 
Sure the degraded level at which a system is operating. 


Numerous potential applications are also listed. 


AE OTENTIAL, BENEFITS OF SMT 

Standardization is one of the basic principles upon which 
the SMT concept was founded and it is the key to the high 
degree of success that has been achieved during the initial 
phase of development. The standardization principles employed 
in the development of the SMT concept should also be applied 
to the development of new equipment by Navy laboratories and 
commercial contractors to ensure that the same techniques are 
used to establish performance standards. Standards of per- 
formance, based on SMT requirements, should be an integral 
part of a system, sub-system, equipment and З de- 
sign specifications. All equipment procurred, as a result of 
КИЛО СОС ОЛ contract, pshoutaRcomply wwth "the! performance 
criteria established by SMT as the minimum acceptable for use 


in an operational environment. 
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As a result of the use of SMT procedures in evaluating 
systems performance, standardization of test procedures would 
be universal regardless of the system configuration or site 
location. Any manager, operator, or supplier should be aware 
of the benefits of standardization. 

When the operators and industry engineers select a common 
unit of measurement, as the EDR, RLMG and SMT procedures can 
provide for communications, everyone will speak the same 
language. Definitive specifications can be stated, and under- 
stood and construction can be monitored in terms of quantified 
parameters, not qualified criteria as is often the practice at 
present. The operator can plan on achieving and holding a 
performance level, which will permit him to make meaningful 
judgments as to his needs and capabilities. 

Once a common unit of measurement has been selected all 
communications systems can be compared and contrasted with 
each other. It is quite certain that duplication of communi- 
cation systems exists between members of the Armed Forces, 
numerous other government agencies, and within industry. 
Duplication is taken to mean two systems which perform the 
S ame tabk ог Similar tasks and for which research and develop- 
ment costs, learning curve costs and implementation costs 
were duplicated. If all concerned agencies had used one mea- 
surement scale it would have been possible to prevent the 
needless additional expense of developing and/or building 


and/or introducing the same system twice. 
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With a test procedure such as SMT, communications systems 
manpower requirements could be reduced in several ways. 
Since an entire external communications system can be checked 
in 5% to 20% of the time needed by present test procedures, 
less personnel are needed to perform the tests. Less down 
time is needed to perform maintenance. Systems and components 
can be tested while installed which means equipment does not 
have to be removed and replaced and it is not exposed to 
damage while handling. Additionally, only defective com- 
ponents need be removed and a better indication of the failure 
can be provided greatly easing the maintenance requirement 
which means less personnel are needed to perform overhaul or 
depot level maintenance. 

The SMT testing procedures will enable the user to deter- 
mine the level of performance of his system. The user will 
be aware of any degradation of the system's performance. He 
Der able. to measures the level of degradation and make a 
decision, based upon quantitative information, regarding his 
ability to complete an assigned mission. 

Proper utilization of the SMT techniques will increase 
the Berman nei ore systems availability. Since availability 
can be achieved through maintainability and maintainability 
needs an adequate test procedure and SMT is, not only, an 
adequate test procedure, but an improvement over any other 
available test procedure, system availability will increase 
which means the system operator and user will derive a benefit 


from utilization of the Standard Measurement Technique. Since 


T 





SMT measures degraded mode operations, maintenance can be 
planned in anticipation of a catastrophic failure which means 
the system should not, as often, suddenly be put out of 
commission. When catastrophic failure does occur, SMT will 
permit quicker trouble shooting and permit a faster restoration 
of the failed system or component. 

Numerous additional bene fits~ may be derived by implemen- 
tation of SMT. When a garbled message is received, the 
receiver could test his system to ensure it is operating 
within specifications. He could inform the sender of the 
garbled message and the sender could do likewise. It may 
even be advantageous to have the SMT test signal broadcast 
from one unit to another in order to test the communications 
system on a macro level. 

An individual unit could introduce a series of messages 
into its own receiver system and then observe how the messages 
are handled by its local message distribution system. 

Taking this idea one step further could enable SMT to 
Make its greatest contribution to Naval Communications. Since 
SMT can measure the level of performance of a communications 
ECC X on a ship, and assign a quantitative value to it, 
it could assign a quantitative value to a total ship's 
communications suit by testing*all on-board communications 
systems and summing their level of performance. A method of 
weighting the various systems would have to be devised. In 
this manner wt could be stated that ship X can communicate 


LEGE ОЗС СЕ о статс согЕтоп with a 7; (959), confidence 


78 





interval. Once this is done every ship in the Navy could have 
a quantitative value (s) assigned to its communications suit. 
Weak performers could receive more attention and strong per- 
formers checked to See what beneficial practices could be 
utilized by other vessels. The same concept could be applied 
to all Naval Communications Stations whereby their ability to 
communicate could be quantified. Once ships and stations 
levels of performance have been determined the test could be 
applied from the communications stations to the ship and vice 
versa by transmitting the test tape. This step would enable 
the performance level of the trunk to be quantified. 

Up to this point we have addressed equipment orientated 
applications. It 15 possible to include personnel, message 
handling, distribution and management. By uSing test tapes 
with a pre-determined sequence of messages introduced at the 
communications ‘station, the flag ‘ship, or at an individual 
ship's receiver, the ability of a communications system to 
put a message in the hands of the ultimate, intended user 
could be evaluated. ГЕНШЕ tape could include messages dealing 
Ыл шорта топы, administration, supply, ete., and range from 
FLASH to ROUTINE in priority. In this manner, all aspects 
of the total communications system, i.e., equipment, personnel, 
management, etc., could be tested and weak points identified. 
The test could be repeated at any location or unit and permit 
a valid, equal comparison between two or more communications 


systems. 
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Test tapes could be transmitted via satellite in order 
to determine the level of performance at which various 
satellite channels are operating. 

Since there is no major communications system test avail- 
able, SMT would complement present sub-system and component 
tests. Some tests will be redundant and no longer needed and 
they can be removed from the inventory producing a cost 
Savings. However the two primary benefits of SMT must not be 
overshadowed. First it is a means to check a communications | 
system, not components, and secondly it provides a means to 
measure degraded performance. Since very few systems are 
operating at full performance levels, an ability to measure 
the degraded level of performance is necessary. 

Retrofit installation of a system and/or equipment is 
an expensive process. The early acceptance of SMT as a 
Standard installation for new construction could result in a 
considerable savings. As an interim measure, consideration 
should be given to the installation of sensors required for 
SMT in new construction pending a decision to include SMT in 


Еа outfitting. 


Dee OTENTI ARS APPLICATIONS OF SMT 

The potential application of Standard Measurement Tech- 
niques (SMT) to electronic systems, other than afloat/ashore 
communication systems, 15 based on the documented successful 
achievement of initial objectives obtained in the development 
of SMT as an integral part of the Navy's overall System 


Performance Measurement Program. 
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The application of the SMT engineering concept to the 
communication links of Tactical Data Systems such as NTDS, 
ATDS, and MTDS, is a natural extension of the initial appli- 
Cation ode SMT. Testing hardware in TDS equipped aircraft 
(Е2С, РЗС, Е14) using real world, simulated inputs, and em- 
ploying SMT procedures for determining the performance of 
Gata links prior to being airborne, would be valuable. The 
SMI concept could also be employed while airborne to monitor 
data link performance, determine source of data link problems 
(own aircraft or termination station), and isolate equipment 
that is operating below established performance levels. 

The techniques of SMT are universal in application to 
EEccLtroude systems we Although, the initial application is 
TOG testing naval communication systems, SMT is applicable to: 

a. Tactical Data Systems 

ТЕ АУТОТТТС ета 

с. ASW Systems 

d. Weapons Systems 

e. interior Communication Systems (ICS) 

Е. Command and Control Systems 

а. Navigation Systems 

h. Aircraft Voice Systems 

i. Submarine Communications Systems 

Ы- ECM Systems 

Plans have been drawn up to "Hard Wire" the SMT system 
шеф ene Communications system of the USS NIMITZ (CVAN-68) 
[Вала into tie fPetrol Frigate (PF) class which is in the 


development stage [39]. 
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КІ CONCEUSTON 


The need for a Naval Communications System Level Test 
Procedure exists. Until this need is satisfied, personnel 
operating the system will not know or be able to use all the 
potential of the system which they are working with. Once 
the level of performance of a system can be quantified, using 
meaningful criteria, such as, the noui degradation which 
it can accept and still be usable for communications or what 
percentage of the system is operating above a predetermined 
level, the managers of the system can determine how much 
through-put the system can provide. This knowledge, capa- 
bility and availability, is the first step towards efficient 
utilization of present and future resources. A test pro- 
cedure is needed which can provide this knowledge. 

It has been shown that all existing test procedures are 
inadequate and mislabeled. Virtually all are called systems 
level test procedures when they are really subsystem level 
procedures measuring output power, percentage modulation, 

SUE | 

A system level test procedure should be able to accomplish 
the following: 

1) Quantify the level of performance of the system or 
determine the present level of degradation at which the sys- 
tem 1S operating. 

2) Determine how much degradation in the input signal a 


system can accept and still put out an error-free or acceptable 
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3) Determine the communications system ability to 
communicate; its reason for being. 

The Standard Measurement Technique (SMT), presently under 
development by the Naval Electronic Systems Command, Southwest 
Division, is a system level test which can fulfill this need. 
By inputing signals of known degradation and quantifying the 
output it can measure the ability of a system to pass infor- 
mation. However, SMT is not an end-all. It does not provide 
all the information which a manager needs in order to operate 
his system. It should be used та compliment some present sub- 
systems level procedures while others should be deemphasized. 
SMT can be an excellent tool in the hands of the operators 
and when used in conjunction with QMS can provide a complete, 
instantaneous systems status report. It can be used to 
point out where the maintenance effort should be applied. 
PMS/3M and Pomsee should be applied as maintenance procedures 
to failed components located by SMT and QMS. 

SMI development, by Naval Electronic Systems Command, 
must be completed. Alternate cost effective pieces of test 
equipment must be found. Test procedures must be firmed and 
lle snake applications must be examined. The SMT validity 
interval needs to be determined so that a decision can be made 
which will evaluate the costs and benefits of applying SMT 
to operational units on a periodic schedule, such as once a 
Pr ter sorsberoregatcrüsep or having te built into all units 


SO it can be used continuously. 
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The Standard Measurement Technique has the potential to 
deliver system level testing and provide Naval Communications 


with a complete, true, system, integrated test structure. 
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APPENDIX A 


Major Test Equipment Functions 


The functions of the major test equipment used in the 

system test are as follows: 

a. Magnetic tape deck and interface buffer used to in- 
put prerecorded standard test message tapes. 

b. Spectrum Analyzer is used to measure the individual 
channel levels of a composite tone, i.e., distortion 
levels, inter-modulation. 

c. Dummy load to absorb the full power transmit signal 
to acceptable testing levels. 

d.  Attenuators used to reduce the receiver input voltage 
levels to acceptable testing levels. 

e. Micro-voltmeter used to measure voltage level at 


receiver. 
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